Imaging apparatus having switching drive modes, imaging system, and mobile object

ABSTRACT

An imaging apparatus performs a global electronic shutter operation. During an exposure period for acquiring one frame, the imaging apparatus transfers electric charges accumulated in a first period from a photoelectric conversion portion to a holding portion. When a second period has elapsed since an end time of the first period, the holding portion holds both electric charges generated in the first period and electric charges generated in the second period. A plurality of pixels included in the imaging apparatus includes a first pixel and a second pixel each having a different saturation charge quantity of the photoelectric conversion portion included in each pixel.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation of U.S. application Ser. No.16/394,765, filed Apr. 25, 2019, which claims the benefit of JapanesePatent Application No. 2018-085872, filed Apr. 26, 2018, which arehereby incorporated by reference herein in their entireties.

BACKGROUND Field of the Invention

The present disclosure relates to an imaging apparatus, an imagingsystem, and a mobile object.

Description of the Related Art

In recent years, a global electronic shutter operation by acomplementary metal oxide semiconductor (CMOS) image sensor has beenproposed. In an imaging apparatus that performs the global electronicshutter operation, an electric charge accumulation operation in thephotoelectric conversion for acquiring one frame starts and endssimultaneously among pixels (exposure start time is the same amongpixels and exposure end time is also the same among pixels). Using aglobal electronic shutter operation is advantageous in that a subjectimage is not distorted even when capturing a quickly moving subject.

Japanese Patent Application Laid-Open No. 2015-177349 discusses animaging apparatus having a global electronic shutter function. In theimaging apparatus discussed in Japanese Patent Application Laid-Open No.2015-177349, in a first period which starts at an exposure start time inan exposure period for acquiring one frame, electric charges generatedby the photoelectric conversion in the first period are accumulated in aphotoelectric conversion portion. At the exposure start time, a holdingportion holds electric charges accumulated in an exposure period foracquiring the preceding frame. In the first period, an operation forreading the electric charges is sequentially performed for each pixel.At the end of the first period, electric charges are transferred fromthe photoelectric conversion portion to the holding portion in eachpixel. In a second period following the first period, electric chargesgenerated by the photoelectric conversion in the second period areaccumulated by the photoelectric conversion portion or by both thephotoelectric conversion portion and the holding portion.

In the imaging apparatus discussed in Japanese Patent ApplicationLaid-Open No. 2015-177349, electric charges are transferred in this wayfrom the photoelectric conversion portion to the holding portion duringan exposure period for acquiring one frame. This configurationimplements a global electronic shutter operation for improving thesaturation charge quantity of pixels while preventing degradation ofimage quality.

SUMMARY

According to an aspect of the present disclosure, an imaging apparatusincludes a plurality of pixels each including a photoelectric conversionportion configured to accumulate electric charges generated by incidentlight, a holding portion configured to hold the electric charges, anamplification portion configured to output a signal based on theelectric charges, a first transfer switch configured to transfer theelectric charges from the photoelectric conversion portion to theholding portion, and a second transfer switch configured to transfer theelectric charges from the holding portion to the amplification portion,and. The imaging apparatus includes output lines connected to theplurality of pixels. At a first time, the photoelectric conversionportions of the plurality of pixels start accumulating the electriccharges. From the first time to a second time, the first transfer switchof at least one of the plurality of pixels is kept OFF, and thephotoelectric conversion portion of the at least one of the plurality ofpixels accumulates electric charges generated in a first period thatstarts at the first time and ends at the second time. The first transferswitches of the plurality of pixels are controlled to turn ON from OFFby the second time at the latest. At a third time following the secondtime, the holding portions of the plurality of pixels hold both electriccharges generated in the photoelectric conversion portion in the firstperiod and electric charges generated in the photoelectric conversionportion in a second period that starts at the second time and ends atthe third time. The plurality of pixels includes a first pixel and asecond pixel. A saturation charge quantity of the photoelectricconversion portion of the first pixel is different from a saturationcharge quantity of the photoelectric conversion portion of the secondpixel.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagingapparatus.

FIG. 2 is a diagram illustrating an equivalent circuit of the imagingapparatus.

FIG. 3 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus.

FIG. 4 is a timing chart illustrating drive pulses of the imagingapparatus in a first drive mode.

FIGS. 5A to 5F are schematic graphs illustrating potential states ofeach portion of a pixel of the imaging apparatus in the first drivemode.

FIG. 6 is a diagram illustrating drive pulses of the imaging apparatusin the first drive mode.

FIG. 7 is a diagram illustrating drive pulses of the imaging apparatusin a second drive mode.

FIGS. 8A to 8E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 9A to 9E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 10A to 10E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 11A to 11E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 12A to 12E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 13A to 13E are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIG. 14 is a diagram illustrating an equivalent circuit of the imagingapparatus.

FIG. 15 is a diagram illustrating an equivalent circuit of the imagingapparatus.

FIG. 16 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus.

FIG. 17 is a diagram illustrating an equivalent circuit of the imagingapparatus.

FIG. 18 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus.

FIGS. 19A to 19D are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 20A to 20D are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIGS. 21A to 21D are schematic diagrams illustrating drive pulses of theimaging apparatus and examples of electric charge quantities in pixelportions in the second drive mode.

FIG. 22 is a block diagram illustrating a configuration of an imagingsystem.

FIGS. 23A and 23B are block diagrams illustrating configurations of amobile object.

DESCRIPTION OF THE EMBODIMENTS

In an imaging apparatus discussed in Japanese Patent ApplicationLaid-Open No. 2015-177349, in a case where the light quantity largelychanges during an exposure period for acquiring one frame, for example,when an image of a quickly moving subject is captured, the output ratioof a signal output from each pixel may deviate from the appropriateoutput ratio. Therefore, there has been a problem of degradation ofimage quality. According to some exemplary embodiments of the presentdisclosure, it is possible to improve image quality.

The imaging apparatus according to an exemplary embodiment of thepresent disclosure is provided with a plurality of pixels, and outputlines connected to the plurality of pixels. Signals from the pluralityof pixels are output to the output lines. Each of the plurality ofpixels includes a photoelectric conversion portion, a holding portionfor storing electric charges, and an amplification portion foroutputting a signal based on electric charges. Each pixel furtherincludes a first transfer switch for transferring electric charges fromthe photoelectric conversion portion to the holding portion, and asecond transfer switch for transferring electric charges from theholding portion to the amplification portion. Such a configurationenables performing an imaging operation in which the photoelectricconversion period is identical between the plurality of pixels, i.e., aglobal electronic shutter operation. The electronic shutter operationrefers to electrically controlling the accumulation of electric chargesgenerated by incident light.

According to some exemplary embodiments of the present disclosure, thephotoelectric conversion portions of the plurality of pixelssimultaneously start accumulating electric charges at the first time(start the exposure period of the kth frame, where k is a positiveinteger equal to or larger than 2). From the first to the second time,the first transfer switch remains OFF in at least one pixel. In the atleast one pixel, electric charges generated in this period areaccumulated in the photoelectric conversion portion. The period from thefirst to the second time is the first period.

In the first period, the amplification portion sequentially outputssignals based on electric charges held in the holding portions of theplurality of pixels to the output line. In other words, each pixeloutputs a signal at least once in the first period. More specifically,in the first period, the second transfer switches of the plurality ofpixels sequentially turn ON. Electric charges generated in the firstperiod are accumulated in the photoelectric conversion portion.Therefore, in the first period, the holding portion can hold electriccharges generated before the first time. More specifically, electriccharges held by the holding portion in the first time are electriccharges generated by the photoelectric conversion portion in theexposure period of the preceding frame, i.e., the (k−1)th frame.

The number of signals output in the first period may be changeddepending on the format of the image to be output. For example, whencapturing a moving image, signals needs to be output for the number ofhorizontal lines used for one frame. According to such an exemplaryembodiment, signals need to be output from not all the pixels includedin an imaging apparatus.

After signals from the plurality of pixels have been output, the holdingportions of the plurality of pixels hold electric charges in at leastthe second period from the second to the third time. At this timing,each holding portion holds electric charges generated in the firstperiod and electric charges generated in the second period. At the thirdtime, the control unit 3 controls turning OFF of the first transferswitches of the plurality of pixels from ON at the same time.

The photoelectric conversion portion needs to accumulate electriccharges generated in at least the first period and therefore is able tomaintain the saturation charge quantity for the pixel even if asaturation charge quantity of the photoelectric conversion portion issmall. This configuration makes it possible to perform the globalelectronic shutter operation while the saturation charge quantity ismaintained. According to some exemplary embodiments of the presentdisclosure, the second period during which the holding portions of theplurality of pixels hold electric charges is longer than the firstperiod. This is because, since the second period is longer than thefirst period, the saturation charge quantity of the photoelectricconversion portion can be reduced.

According to some exemplary embodiments of the present disclosure, animaging apparatus which performs the global electronic shutter operationfor transmitting electric charges from the photoelectric conversionportion to the holding portion during the exposure period for acquiringone frame can prevent the deviation of the pixel output ratio.

Exemplary embodiments of the present disclosure will be described belowwith reference to the accompanying drawings. The exemplary embodimentsaccording to the present disclosure are not limited only to thefollowing exemplary embodiments. For example, an example case where apart of the configuration of any one of the following exemplaryembodiments is added to other exemplary embodiments, and an example casewhere a part of the configuration thereof is replaced with a part of theconfiguration of other exemplary embodiments are also included inexemplary embodiments of the present disclosure. According to thefollowing exemplary embodiments, a first conductivity type is the Ntype, and a second conductivity type is the P type. However, the firstconductivity type may be the P type, and the second conductivity typemay be the N type.

A first exemplary embodiment will be described below. FIG. 1 is a blockdiagram illustrating a configuration of the imaging apparatus. Animaging apparatus 1 can be configured by one chip by using asemiconductor substrate. The imaging apparatus 1 includes an imagingregion 2 where a plurality of pixels is arranged. The imaging apparatus1 further includes a control unit 3 for supplying control signals andpower voltages to a vertical scanning unit 4, a signal processing unit5, and an output unit 6.

The vertical scanning unit 4 supplies drive pulses to a plurality ofpixels arranged in the imaging region 2. Normally, the vertical scanningunit 4 supplies drive pulses for each pixel row or for each of aplurality of pixel rows. The vertical scanning unit 4 can be configuredby using shift registers or address decoders.

The signal processing unit 5 includes a column circuit, a horizontalscanning circuit, and a horizontal output line. The column circuitincludes a plurality of circuit blocks for receiving signals of aplurality of pixels included in a pixel row selected by the verticalscanning unit 4. Each circuit block can include any one of a signalholding portion, an amplification circuit, a noise rejection circuit,and an analog-to-digital conversion circuit, or a combination thereof.The horizontal scanning circuit can be configured by using shiftregisters or address decoders.

The output unit 6 outputs the signal transmitted via the horizontaloutput line, to the outside of the imaging apparatus 1. The output unit6 includes a buffer or an amplification circuit and outputs a signalfrom each pixel to the signal processing unit 7. The signal processingunit 7 performs signal processing on the signal from each pixel.Typically, the signal processing unit 7 performs processing forgenerating a developed image by multiplying the signal from each pixelby the gain according to the white balance. The signal processing unit 7may be included in the imaging apparatus 1.

(Pixel Structure)

FIG. 2 is a diagram illustrating an equivalent circuit of a pixelincluded in the imaging region 2 of the imaging apparatus 1. To simplifydescriptions, it is assumed that the imaging region 2 includes ninepixels arranged in a 3-by-3 matrix form including an nth to an (n+2)throws and an mth to an (m+2)th columns One pixel is enclosed in a dashedline and is referred to as PIX. Although nine pixels PIX are illustratedin FIG. 2 , the imaging apparatus 1 includes a larger number of pixels.

Each pixel PIX includes a photoelectric conversion portion PD, a holdingportion MEM, an amplification portion SF, a first transfer switch TX1,and a second transfer switch TX2. Each pixel PIX further includes areset transistor RES and a selection transistor SEL.

The photoelectric conversion portion PD accumulates electric chargesgenerated by incident light. The anode of the photoelectric conversionportion PD is grounded to a fixed potential, and the cathode thereof isconnected to one terminal of the holding portion MEM via the firsttransfer switch TX1 as a first transfer portion. The first transferswitch TX1 transfers electric charges in the photoelectric conversionportion PD to the holding portion MEM. The cathode of the photoelectricconversion portion PD is connected to a power supply line as a secondpower supply which functions as an overflow drain (hereinafter referredto as an OFD) via a third transfer switch TX3 as a third transferportion. The third transfer switch TX3 is a discharge switch fordischarging electric charges in the photoelectric conversion portion PD.

The other terminal of the holding portion MEM is grounded to a fixedpotential. One terminal of the holding portion MEM is also connected tothe gate terminal of the amplification transistor as the amplificationportion SF via the second transfer switch TX2 as a second transferportion. The second transfer switch TX2 transfers electric charges inthe holding portion MEM to a floating diffusion portion FD as the inputnode of the amplification portion SF. The gate terminal of theamplification portion SF is connected to a pixel power supply line as afirst power supply via the reset transistor RES as a reset portion. Thereset transistor RES resets the voltage of the floating diffusionportion FD as the input node of the amplification portion SF. Theselection transistor SEL selects the pixel PIX which outputs a signal toa vertical signal line OUT as an output line. The amplification portionSF outputs a signal based on electric charges generated by incidentlight to the vertical signal line OUT. The amplification portion SF is,for example, a source follower circuit. Each of the first transferswitch TX1, the second transfer switch TX2, and the discharge switch TX3is a metal oxide semiconductor (MOS) transistor. Referring to FIG. 2 ,the pixel power supply line suppling the power voltage for theamplification portion SF and the power supply line which functions as anOFD are separated. However, they may be collectively replaced with acommon power supply line.

One main electrode of the selection transistor SEL is connected to thevertical signal line, and the other main electrode thereof is connectedto one main electrode of the amplification portion SF (transistor). Whenan active signal PSEL is input to the control electrode of the selectiontransistor SEL, both main electrodes of the selection transistor SELenter the conducting state. The amplification portion SF and aconstant-current source (not illustrated) disposed on the verticalsignal line OUT form a source follower circuit. A signal according tothe potential of the gate terminal (control electrode) of theamplification portion SF appears on the vertical signal line OUT. Asignal is output from the imaging apparatus 1 based on the signalappearing on the vertical signal line OUT, i.e., the signal output fromthe pixel PIX. The floating diffusion portion FD as an input node iscommonly connected with the gate terminal of amplification portion SFand the main electrodes of the reset transistor RES and the secondtransfer switch TX2. Therefore, the floating diffusion portion FD has acapacitance value and is capable of storing electric charges.

The imaging apparatus 1 according to the present exemplary embodiment isa color sensor in which each pixel PIX is provided with a color filter.As illustrated in FIG. 2 , pixels PIX with red (R), green (Gr, Gb), andblue (B) color filters included therein are arranged in Bayerarrangement, with the color of the color filter indicated at the upperleft for each pixel. In the imaging apparatus 1 according to the presentexemplary embodiment, each row is provided with two different gate drivelines PTX3-1 and PTX3-2 of the discharge switch TX3. The red (R), green(Gr, Gb), and blue (B) pixels PIX are connected to different drive linesby color of the color filters.

According to the present exemplary embodiment, in an nth row, the gatedrive line PTX3-1(n) is connected to the red (R) pixels PIX, and thegate drive line PTX3-2(n) is connected to the green (Gr) pixel PIX. Inan (n+1)th row, the gate drive line PTX3-1(n+1) is connected to thegreen (Gb) pixels PIX, and the gate drive line PTX3-2(n+1) is connectedto the blue (B) pixel PIX. The gate drive line PTX3-2(n) for driving thegreen (Gr) pixels PIX and the gate drive line PTX3-1(n+1) for drivingthe green (Gb) pixels PIX are identical in a region (not illustrated).

FIG. 3 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus. More specifically, FIG. 3 illustrates across-section of one pixel. Referring to FIG. 3 , portions having thesame function as the portions illustrated in FIG. 2 are assigned thesame reference numerals. Although FIG. 3 illustrates a front-sideilluminated imaging apparatus, the imaging apparatus is not limitedthereto and may be a back-side illuminated imaging apparatus. Inaddition, the conductive type of each semiconductor region will bedescribed below centering on an example case where electrons are used assignal electric charges. When using holes as signal electric charges,the conductive type of each semiconductor region needs to be theopposite conductivity type.

The photoelectric conversion portion PD has an embedded photo-diodestructure. The photoelectric conversion portion PD includes an N-typesemiconductor region 202 and a P-type semiconductor region 203. TheN-type semiconductor region 202 and the P-type semiconductor region 203form a PN junction. By the P-type semiconductor region 203, interfacenoise can be prevented.

The P-type semiconductor region 201 is disposed around the N-typesemiconductor region 202. The P-type semiconductor region 201 may beformed by implanting P-type impurity ions into an N-type semiconductorsubstrate or by using a P-type semiconductor substrate.

The holding portion MEM includes an N-type semiconductor region 205.Signal electric charges are held in the N-type semiconductor region 205.A P-type semiconductor region may be disposed between the N-typesemiconductor region 205 and the front face of the semiconductorsubstrate. With such an arrangement, the holding portion MEM has anembedded structure.

A gate electrode 207 configures the gate of the first transfer switchTX1. An N-type semiconductor region 213 is disposed under the gateelectrode 207 via a gate insulation film. The impurity concentration ofthe N-type semiconductor region 213 is lower than the impurityconcentration of the N-type semiconductor region 202. The potentialstate between the photoelectric conversion portion PD and the holdingportion MEM can be controlled by the voltage supplied to the gateelectrode 207.

An N-type semiconductor region 208 configures the floating diffusionportion FD. The N-type semiconductor region 208 is electricallyconnected with the gate of the amplification portion SF via a plug 209.

A gate electrode 204 configures the gate of the second transfer switchTX2. The potential state between the holding portion MEM and thefloating diffusion portion FD can be controlled by the voltage suppliedto the gate electrode 204.

The holding portion MEM is shaded by a light shielding portion 210. Thelight shielding portion 210 is made of a metal such as tungsten,aluminum, and other materials having low transmittance of light in thevisible light region. The light shielding portion 210 covers at leastthe holding portion MEM. As illustrated in FIG. 3 , the light shieldingportion 210 may cover the entire gate electrode 207 of the firsttransfer switch TX1 and may be extended to a part of the upper portionof the gate electrode 204 of the second transfer switch TX2. Thisconfiguration makes it possible to further improve the shieldingfunction of the light shielding portion 210. A color filter CF and amicro lens ML are disposed above the opening of the light shieldingportion 210. More specifically, in each pixel PIX, the color filter CFand the micro lens ML are disposed on the upstream side of thephotoelectric conversion portion PD in the incident direction ofincident light.

An N-type semiconductor region 212 configures a part of an OFD region. Aplug 215 supplies the power voltage to the N-type semiconductor region212 and is connected with the second power supply which functions as theOFD.

A gate electrode 211 configures the gate of the discharge switch TX3.The potential state between the photoelectric conversion portion PD andthe OFD region can be controlled by the voltage supplied to the gateelectrode 211.

(Driving Method)

A method for driving the imaging apparatus will be described below. Adescription will be given of a first drive mode and a second drive modefor the method for driving the imaging apparatus. In both the first andthe second drive modes, the global electronic shutter operationincluding the accumulation of electric charges by the photoelectricconversion portion PD and electric charge transfer from thephotoelectric conversion portion PD to the holding portion MEM isperformed. The difference between the first and the second drive modesis the timing of electric charge transfer. More specifically, in thefirst drive mode, the control unit 3 performs electric charge transferfrom the photoelectric conversion portion PD to the holding portion MEMat the end of the exposure period. In the second drive mode, the controlunit 3 performs electric charge transfer from the photoelectricconversion portion PD to the holding portion MEM during the exposureperiod. The imaging apparatus according to the present exemplaryembodiment operates at least in the second drive mode.

The imaging apparatus according to the present exemplary embodiment mayhave a drive mode for performing a rolling shutter operation. In thedrive mode for the rolling shutter operation, the photoelectricconversion portion PD of each of the plurality of pixels sequentiallystarts accumulating electric charges. Subsequently, the control unit 3sequentially controls turning ON of the first transfer switches TX1 ofthe plurality of pixels.

(First Drive Mode)

The first drive mode for performing the global electronic shutteroperation will be described below with reference to FIGS. 4 to 6 . Inthe first drive mode, the photoelectric conversion portion PDaccumulates electric charges generated in the exposure period, and, atthe end of the exposure period, the electric charges accumulated in thephotoelectric conversion portion PD is transferred to the holdingportion MEM.

FIG. 4 is a diagram schematically illustrating drive pulses in the firstdrive mode. FIG. 4 illustrates drive pulses supplied to the selectiontransistor SEL, the reset transistor RES, the first transfer switch TX1,the second transfer switch TX2, and the discharge switch TX3 in eachpixel in the nth to (n+2)th rows. When a drive pulse is at the highlevel, the corresponding transistor or switch turns ON. When a drivepulse is at the low level, the corresponding transistor or switch turnsOFF. These drive pulses are supplied by the control unit 3 included inthe imaging apparatus 1.

FIGS. 5A to 5F are schematic graphs illustrating potential states, i.e.,potentials to electrons of each portion of a pixel at each timing. Ineach of the graphs, a higher potential to electrons, i.e., a lowerelectric potential, is plotted on a higher end of the graph. FIGS. 5A to5F illustrate potentials of the 01-D, the discharge switch TX3, thephotoelectric conversion portion PD, the first transfer switch TX1, theholding portion MEM, the second transfer switch TX2, and the floatingdiffusion portion FD (see FIGS. 2 and 3) from left to right.

Exposure for the preceding frame is performed before the time t2.Exposure indicates that electric charges generated by the photoelectricconversion are accumulated or held as a signal.

Before the time t1, the photoelectric conversion portion PD isirradiated with light, and electric charges corresponding to the lightquantity of the incident light to the photoelectric conversion portionPD are accumulated in the photoelectric conversion portion PD in theperiod until the time t1. The state is schematically illustrated in FIG.5A.

In the period from the time t1 to the time t2, first transfer switchcontrol signals PTX1(n), PTX1(n+1), and PTX1(n+2) (hereinaftercollectively referred to as “PTX1”) becomes the high level, and thefirst transfer switch TX1 turns ON. Consequently, electric chargesaccumulated in the photoelectric conversion portion PD are transferredto the holding portion MEM. This state is schematically illustrated inFIG. 5B. At the time t2, PTX1 becomes the low level again and thetransfer operation is completed. This state at this timing isschematically illustrated in FIG. 5C. The operation for electric chargetransfer from the photoelectric conversion portion PD to the holdingportion MEM is performed for all the pixels at the same time. To endexposure for the preceding frame in the first drive mode, the controlunit 3 controls turning OFF of the first transfer switch TX1, whichtransfers electric charges from the photoelectric conversion portion PDto the holding portion MEM, from ON for all the pixels at the same time(at the time t2 illustrated in FIG. 4 ).

In the period from the time t3 to the time t4, discharge switch controlsignals PTX3-1 (n), PTX3-1(n+1), PTX3-1 (n+2), and PTX3-2(n) PTX3-2(n+1) and PTX3-2(n+2) (hereinafter collectively referred to as “PTX3”)become the high level, and electric charges are discharged from thephotoelectric conversion portion PD to the OFD. The state isschematically illustrated in FIG. 5D. This operation for dischargingelectric charges from the photoelectric conversion portion PD to the OFDis performed for all the pixels at the same time. To start exposure inthe first drive mode, the control unit 3 controls turning OFF of thedischarge switch TX3 which transfers electric charges from thephotoelectric conversion portion PD to the holding portion MEM, from ONfor all the pixels at the same time (at the time t4 illustrated in FIG.4 ).

In the first drive mode, the control unit 3 discharges electric chargesfrom the photoelectric conversion portion PD to the OFD and transferselectric charges from the photoelectric conversion portion PD to theholding portion MEM for all the pixels at the same time. With thisconfiguration, the imaging apparatus can perform an imaging operation inwhich the photoelectric conversion period is identical between theplurality of pixels, i.e., a global electronic shutter operation.

An operation for reading electric charges transferred from thephotoelectric conversion portion PD to the holding portion MEM will bedescribed below. While a description will be given of a case where, anoperation for reading the signal of the preceding frame is started afterexposure (accumulation of electric charges in the photoelectricconversion portion PD) for the following frame is started at the timingof when the control unit 3 controls turning OFF of the discharge switchTX3 from ON, the operation is not limited thereto.

At the time t5, the control unit 3 controls turning OFF of the resettransistor RES(n) of the floating diffusion portion FD from ON to cancelthe reset state of the floating diffusion portion FD. At the same time,when a row selection control signal PSEL(n) becomes the high level, theselection transistor SEL(n) turns ON, and the pixels in the nth row areselected. Consequently, the voltage corresponding to the potentials ofthe floating diffusion portions FD of the pixels in the nth row isoutput to the vertical signal line OUT.

In the period from the time t6 to the time t7, a reset noise selectionsignal PTN becomes the high level. At this timing, the column circuitincluded in the signal processing unit 5 samples and holds the resetnoise levels of the floating diffusion portions 1-D of the pixels in thenth row. Consequently, a noise signal read operation (N read) isperformed.

In the period from the time t8 to the time t9, the second transferswitch control signal PTX2(n) becomes the high level, and the secondtransfer switches TX2 of the pixels in the nth row turn ON.Consequently, electric charges accumulated in the holding portion MEMare transferred to the floating diffusion portion FD. The state isillustrated in FIG. 5E. At the time t9, the second transfer switchcontrol signal PTX2(n) becomes the low level, and the second transferswitches TX2 of the pixels in the nth row turn OFF. This completes theoperation for electric charge transfer from the holding portion MEM tothe floating diffusion portion FD. This operation completion state isillustrated in FIG. 5F.

At the time t9, the operation for electric charge transfer from theholding portion MEM to the floating diffusion portion FD is completed.Then, a potential having a level (optical signal level) increased byadding the potential corresponding to electric charges accumulated inthe floating diffusion portion FD to the reset level is output to thevertical signal line OUT. In the period from the time t10 to the timet11, an image signal selection signal PTS becomes the high level, andthe column circuit samples and holds this optical signal level.Accordingly, an optical signal read operation (S read) is performed. Thereset noise level previously sampled and held is subtracted from theoptical signal level, and the resultant level is output to thesubsequent stage as an image signal without noise.

At the time t12, the selection transistor SEL(n) turns OFF, the resettransistor RES(n) turns ON, and the potentials of the floating diffusionportions FD of the pixels in the nth row are reset again.

According to the above-described drive pulses, a series of readoperations including pixel selection, reset, noise signal read (N read),electric charge transfer to the floating diffusion portion FD, andoptical signal read (S read) is performed for each pixel row. The outputsignal may be subjected to analog-to-digital (A/D) conversion outsidethe imaging apparatus or inside the imaging apparatus. As illustrated inFIG. 4 , following the n th row, the control unit 3 also sequentiallyperforms the read operation for the pixels in the (n+1)th and (n+2)throws.

According to the present specification, the period during which “noisesignal read (N read), electric charge transfer to the floating diffusionportion FD, and optical signal read (S read)” are performed in eachpixel row is defined as a “read period”. Referring to FIG. 4 , the readperiod on the left-hand side of the timing chart indicates the readperiod for the kth frame, and the subsequent read period on theright-hand side of the timing chart indicates the read period for the(k+1) th frame. According to the present specification, the periodduring which the photoelectric conversion portion PD of each pixelaccumulates electric charges corresponding to one frame is defined as an“exposure period”. According to the present exemplary embodiment, theperiod from the time t4 of when discharge of electric charges to the OFDis completed to the time t13 of when electric charge transfer to theholding portion MEM is completed is the exposure period for the (k+1)thframe. As illustrated in the upper and lower sides of the timing chartillustrated in FIG. 4 , according to the present exemplary embodiment,the read period for the kth frame and the exposure period for the(k+1)th frame are at least partially overlapped with each other.

In the subsequent descriptions, drive pulses supplied to each portion ina pixel to perform a series of operations including “noise signal read(N read), electric charge transfer to the floating diffusion portion FD,and optical signal read (S read)” are collectively referred to as a“read signal”. FIG. 6 is a diagram illustrating drive pulses of theimaging apparatus in the first drive mode. The oblique lines indicatethat read operations are sequentially performed for each row. While thefirst transfer switch control signal PTX1 and the discharge switchcontrol signal PTX3 are supplied for all the pixels at the same time,FIG. 6 illustrates one first transfer switch control signal PTX1 and onedischarge switch control signal PTX3 for all the pixels in a collectiveway. Although FIG. 6 illustrates the same timing as FIG. 4 , the timingillustrated in FIG. 6 is a simplified representation of the timingillustrated in FIG. 4 for use in a case of multi-pixel arrangement.

As described above, in the first drive mode, the control unit 3 performselectric charge transfer from the photoelectric conversion portion PD tothe holding portion MEM at the end of the exposure period. Therefore,electric charge transfer from the photoelectric conversion portion PD tothe holding portion MEM is performed only once during the exposureperiod for one frame. Such a drive is also referred to as one-timetransfer drive.

(Second Drive Mode)

The second drive mode for performing the global electronic shutteroperation will be described below with reference to FIG. 7 . In thesecond drive mode, electric charges generated in the exposure period areaccumulated in the photoelectric conversion portion PD and the electriccharges accumulated in the photoelectric conversion portion PD aretransferred to the holding portion MEM during the exposure period. Afterthe first-time charge transfer to the holding portion MEM, thephotoelectric conversion portion PD or both the photoelectric conversionportion PD and the holding portion MEM accumulate electric chargesgenerated by the photoelectric conversion in subsequent periods. Anexample of the second drive mode will be described below centering on anexample case where electric charge transfer from the photoelectricconversion portion PD to the holding portion MEM is performed twice inthe exposure period for one frame. For operations equivalent to those inthe first drive mode, redundant descriptions will be omitted. Such adrive is also referred to as two-time transfer drive.

FIG. 7 schematically illustrates drive pulses in the second drive mode.Similar to the first drive mode, exposure for the preceding frame isperformed before the time t2, and, at the time t4, electric chargesgenerated by the photoelectric conversion portion PD in the exposureperiod of the preceding frame are accumulated in the holding portionMEM. When the control unit 3 controls turning OFF of the dischargeswitch TX3 from ON at the time t4, an exposure period for the followingframe is started. The time t4 is time when an exposure period isstarted, and is referred to as a first time.

The first transfer switch TX1 remains OFF from the first time (time t4)until the time when the first period elapses, i.e., at the second time(time t14). According to the present exemplary embodiment, while thefirst transfer switches TX1 of all the pixels remain OFF, the presentexemplary embodiment is not limited thereto. Alternatively, in at leastone pixel, the first transfer switch TX1 remains OFF from the first tothe second time.

The time when the first period has elapsed since the first time (timet4) is the second time (time t14). More specifically, the period fromthe first time (time t4) to the second time (time t14) is the firstperiod. In the first period, electric charges generated in the firstperiod are accumulated in the photoelectric conversion portion PD.

In the first period, operations for reading electric chargescorresponding to the preceding frame held in the holding portion MEM aresequentially performed. Read operations are sequentially performed sothat the signal output corresponding to the preceding frame completes bythe second time at the latest (i.e. completes before, or by, the secondtime).

At the second time (time t14), the control unit 3 turns ON the firsttransfer switch TX1. Accordingly, electric charges in the photoelectricconversion portion PD are transferred to the holding portion MEM. Morespecifically, after the second time, electric charges generated in thefirst period are held by the holding portion MEM. According to thepresent exemplary embodiment, the first transfer switches TX1 of all thepixels are turned ON from OFF at the same time. However, the firsttransfer switches TX1 of the plurality of pixels need to turn ON by thesecond time, and the transition timing may be different between thefirst transfer switches TX. For example, the control unit 3 may turn ONthe first transfer switch TX1 in order of pixels in which theabove-described read operation is completed.

The time when the second period has elapsed since the second time (timet14) is the third time (time t13). More specifically, the period fromthe second time (time t14) to the third time (time t13) is the secondperiod. According to the present exemplary embodiment, at the time t15after the time t14, the control unit 3 controls turning OFF of the firsttransfer switch TX1 from ON again. Electric charges generated in thesecond exposure from the time t15 to the time t13 are accumulated in thephotoelectric conversion portion PD. In the second exposure period,electric charges generated in the first exposure period from the time t4to the time t15 are held by the holding portion MEM. When the controlunit 3 controls turning ON of the first transfer switch TX1 from OFFagain at the time between the time t15 and the time t13, electriccharges in the photoelectric conversion portion PD are transferred tothe holding portion MEM. Therefore, at the third time (time t13), theholding portion MEM holds both electric charges generated in the firstperiod and electric charges generated in the second period.

As described above, in the second drive mode, the next exposure can bestarted immediately after completion of exposure for one frame. Thus,periods during which information is missing can be eliminated, and imagequality can be therefore improved.

In the second drive mode, after completion of operations for readingelectric charges corresponding to the preceding frame held in theholding portion MEM, the control unit 3 performs electric chargetransfer from the photoelectric conversion portion PD to the holdingportion MEM during the exposure period. Consequently, the saturationcharge quantity of pixels can be increased even with a small saturationcharge quantity of the photoelectric conversion portion PD. Thesaturation charge quantity of pixels refers to the maximum value of theelectric charge quantity which can be handled as a signal from amongelectric charges generated in a single exposure operation. Thesaturation charge quantity of the photoelectric conversion portion PDrefers to the maximum value of the electric charge quantity which can beaccumulated by the photoelectric conversion portion PD. The saturationcharge quantity of the holding portion MEM refers to the maximum valueof the electric charge quantity which can be held by the holding portionMEM.

A single exposure period is the sum total of the first and the secondperiods. Since electric charges of the preceding frame held by theholding portion MEM are read in the first period, electric charges canbe held by the holding portion MEM when the first period ends. Thus, thephotoelectric conversion portion PD needs to accumulate electric chargesgenerated at least in the first period. Normally, since the amount ofelectric charges generated in the first period is less than the amountof electric charges generated in a single exposure time, the saturationcharge quantity of the photoelectric conversion portion PD can bereduced.

According to the present exemplary embodiment, in the second period, thecontrol unit 3 controls turning OFF of the first transfer switch TX1from ON only once (time t15). However, control is not limited thereto.In the second period, the control unit 3 may control turning OFF of thefirst transfer switch TX1 from ON zero times or a plurality of times.When turning OFF of the first transfer switch TX1 is performed zerotimes, electric charges generated in the photoelectric conversionportion PD in the second exposure are immediately transferred to theholding portion MEM and held by the holding portion MEM. When turningOFF of the first transfer switch TX1 is performed a plurality of times,electric charge transfer from the photoelectric conversion portion PD tothe holding portion MEM is performed at least three times in theexposure period for one frame. If the control unit 3 controls turningOFF of the first transfer switch TX1 from ON a plurality of times in thesecond period, the saturation charge quantity of the photoelectricconversion portion PD can be further reduced.

It is desirable that the number of times of turning ON the firsttransfer switch TX1 from OFF in the exposure period corresponding to oneframe is almost equal to or larger than the ratio of the saturationcharge quantity of the holding portion MEM to the saturation chargequantity of the photoelectric conversion portion PD. It is moredesirable that the number of times of turning ON the first transferswitch TX1 is almost equal to the ratio. With this configuration, thesizes of the photoelectric conversion portion PD and the holding portionMEM can be optimized.

(Effects of Present Exemplary Embodiment)

A configuration and effects of the present exemplary embodiment will bedescribed below with reference to FIGS. 8A to 13E. A description will begiven of a case where the imaging apparatus 1 operates in the seconddrive mode.

When an image is captured using a camera, a subject is illuminated by acertain light source, and the light of the light source is reflected bythe subject and is incident to the imaging apparatus 1 through a lens.FIGS. 8A to 13E illustrate an example case where a subject perceived aswhite by the human eyes is captured under the light source.

First Comparative Example

Before describing the configuration and effects of the present exemplaryembodiment, a comparative example will be described below. In theimaging apparatus according to a first comparative example, thesaturation charge quantity of the photoelectric conversion portion PD isidentical for each of the red (R), green (Gr, Gb), and blue (B) pixels(hereinafter referred to as a “R pixel”, “G pixel”, and “B pixel”,respectively). FIGS. 8A to 10E illustrate operations of the imagingapparatus according to the first comparative example under differentincident light conditions.

FIG. 8A is a timing chart illustrating a drive pulse in a similartwo-time transfer operation to that illustrated in FIG. 7 . FIG. 8Billustrates the light quantity of light incident to a certain region ofthe imaging apparatus. Referring to FIG. 8B, the lower dashed lineindicates the light quantity 0 (no incident light), and the upper dashedline indicates the light quantity with which the photoelectricconversion portion PD of the G pixel just reaches the saturation in eachof the first and the second exposure periods. Referring to the exampleillustrated in FIGS. 8A to 8E, light of an intermediate quantity betweenthe two dashed lines illustrated in FIG. 8B is incident to the region inthe imaging plane. The light quantity is constant throughout the firstand the second exposure periods illustrated in FIGS. 8A to 8E.

FIGS. 8C, 8D, and 8E illustrate the changes with time of the electriccharge quantities in the photoelectric conversion portions PD of the G,R, and B pixels, respectively. For example, the operation of the G pixelwill be described below.

At the time t4, when the discharge switch TX3 turns OFF from ON, theaccumulation of electric charges in the photoelectric conversion portionPD is started. In the example illustrated in FIGS. 8A to 8E, since theincident light quantity is constant, the electric charge quantity of thephotoelectric conversion portion PD increases in proportion to time. Atthe time t15, the first-time charge transfer from the photoelectricconversion portion PD to the holding portion MEM is completed, and theelectric charge quantity of the photoelectric conversion portion PDbecomes zero.

In the second exposure period starting from the time t15, the electriccharge quantity of the photoelectric conversion portion PD similarlyincreases in proportion to time. At the time t13, electric charges aretransferred again to the holding portion MEM, and the electric chargequantity of the photoelectric conversion portion PD becomes zero again.

If the saturation level of the photoelectric conversion portion PD is 1,in the example illustrated in FIGS. 8A to 8E, an electric chargequantity of 0.7 is accumulated in the G pixel in each of the first andthe second exposure periods. These electric charge quantities are addedin the holding portion MEM, and the G pixel outputs an electric chargequantity of 0.7+0.7=1.4 for the frame.

In this case, the R and B pixels also perform a similar operation. Inthe examples illustrated FIGS. 8A to 8E, an electric charge quantity of0.6+0.6=1.2 is accumulated in the R pixel, and an electric chargequantity of 0.35+0.35=0.7 is accumulated in the B pixel.

In the example illustrated in FIGS. 8A to 8E, the ratio of the lightquantities (G:R:B) when a subject perceived as white by the human eyesis captured under the light source is 1.4:1.2:0.7. This ratio depends onthe light source with which the subject is irradiated and is referred toas a white balance.

Supplementary information of the white balance will be provided below.The human eyes perceive a white object as white under any light source.To meet this condition of the human eyes, a camera multiplies each ofthe signals from the G, R, and B pixels by the gain according to thewhite balance to generate a final developed image. This gain is referredto as a white balance gain. The white balance gain is normallymultiplied by an engine (signal processing unit) in the subsequent stageof the imaging apparatus. In this example, for a subject perceived aswhite by the human eyes under this light source, the engine multipliesthe signal from the R pixel by 1.4/1.2=1.17 and multiplies the signalfrom the B pixel by 1.4/0.7=2 to equalize the luminance values of the G,R, and B pixels in the final developed image. If the white balance isinappropriate, there arises a phenomenon in which a white object is notperceived as white in the final developed image. In this example, thisphenomenon is referred to as a coloring problem.

According to the first comparative example, as described above, thephotoelectric conversion portions PD of the G, R, and B pixels providethe same saturation charge quantity. According to the first comparativeexample, the OFF level of the gate drive line PTX3 of the dischargeswitch TX3 is common to all the pixels. More specifically, all thepixels provide the same height of the potential barrier of the dischargeswitch TX3 in the OFF state. Even if all the pixels provide the samesaturation charge quantity as in the first comparative example, thecoloring problem does not arise if none of the photoelectric conversionportions PD of the pixels reach the saturation in the exposure period,as illustrated in FIGS. 8A to 8E.

FIGS. 9A to 9E illustrate the electric charge quantity of each pixel ina case where the incident light quantity is larger than the incidentlight quantity in the case illustrated in FIGS. 8A to 8E in the imagingapparatus according to the first comparative example. FIGS. 9A to 9Eillustrate a case where the incident light quantity to the region ishigher than the saturation level of the G pixel, and the light quantityis constant in the one-frame period.

At this timing, the electric charge quantity of the photoelectricconversion portion PD of the G pixel reaches the saturation level 1 inboth the first and the second exposure periods. Since the electriccharge quantity equal to or larger than the saturation charge quantitycannot be accumulated in the photoelectric conversion portion PD, the Gpixel outputs an electric charge quantity of 1+1=2 for the frame.

As illustrated in FIGS. 9D and 9E, when the G pixel reaches thesaturation level “1” although neither of the R and B pixels of theregion have reached the saturation, the engine in the subsequent stagedetermines that the output of the region is saturated. In such a region,for example, all of the G, R, and B pixels are determined to have themaximum gradation. Therefore, the region is processed as a white regionwith the maximum luminance in the final developed image.

Therefore, the coloring problem does not arise even if all the pixelsprovide the same saturation charge quantity as in the first comparativeexample also in a case where the photoelectric conversion portion PD ofthe G pixel reaches the saturation in both the first and the secondexposure periods as illustrated in FIGS. 9A to 9E.

FIGS. 10A to 10E illustrate a situation where the coloring problemarises in the imaging apparatus according to the first comparativeexample. FIGS. 10A to 10E indicates a case where, for example, a subjectperceived as white by the human eyes is moving. In this case, theincident light quantity to the region exceeds the G saturation level inthe first exposure period, and the incident light becomes darker toprovide an intermediate light quantity in the second exposure period.

The electric charge quantity of the photoelectric conversion portion PDof the G pixel reaches the saturation level 1 in the first exposureperiod, and becomes 0.7 in the second exposure period, whereby a finaloutput is 1+0.7=1.7. On the other hand, the photoelectric conversionportions PD of neither of the R and B pixels reach the saturation inboth the first and the second exposure periods.

In this case, since the final output of the G pixel does not reach “1”,the engine in the subsequent stage does not determine the saturation.However, since only the photoelectric conversion portion PD of the Gpixel reaches the saturation in the first exposure period, the electriccharge quantity of the G pixel becomes smaller than the inherent value.Thus, the ratio of the electric charge quantities of the G, R, and Bpixels deviates. As a result, the output ratio between the G, R, and Bpixels for the frame will be deviated.

Under this light source, the appropriate ratio G:R:B when a whitesubject is captured is 1.4:1.2:0.7=1:0.86:0.5 as illustrated in FIGS. 8Ato 8E. However, in the situation illustrated in FIGS. 10A to 10E, theoutput ratio G:R:B is 1.7:1.55:0.9=1:0.91:0.53 which is deviated fromthe appropriate ratio G:R:B. In this case, since the R:B output ratiohas relatively increased, the final developed image will be colored inmagenta. More specifically, in the imaging apparatus according to thefirst comparative example, if a bright subject moves, the periphery ofthe subject may be colored.

A configuration and effects of the present exemplary embodiment will bedescribed below centering on the first exemplary embodiment as anexample. In the imaging apparatus according to the first exemplaryembodiment, the saturation charge quantity of the photoelectricconversion portion PD is differentiated between the G, R, and B pixels.More specifically, the imaging apparatus includes a first pixel, asecond pixel, and a third pixel each having a different saturationcharge quantity of the photoelectric conversion portion PD. Morespecifically, when the saturation charge quantity of the photoelectricconversion portion PD of the G pixel is set to 1, the saturation chargequantity of the photoelectric conversion portion PD of the R pixel isset to 0.86, and the saturation charge quantity of the photoelectricconversion portion PD of the B pixel is set to 0.5. As illustrated inFIG. 2 , this configuration is implemented by separately providing thegate drive line PTX3 for each of the G, R, and B pixels by color anddifferentiating the OFF-level potential of the gate drive line PTX3between the G, R, and B pixels.

This saturation ratio is equalized to the sensitivity ratio of the G, R,and B pixels when a subject perceived as white by the human eyes iscaptured under this light source. In other words, the saturation ratiois equalized to the white balance ratio under the light source.

Like the case illustrated in FIGS. 10A to 10E, FIGS. 11A to 11Eillustrate the electric charge quantity of each pixel in the imagingapparatus according to the first exemplary embodiment in a case wherethe light quantity exceeds the G saturation level in the first exposureperiod and becomes an intermediate light quantity in the second exposureperiod. As described above, the saturation charge quantity isdifferentiated between the G, R, and B pixels according to the whitebalance. Therefore, when the G pixel is saturated in the first exposureperiod, the R and B pixels also reach respective saturation quantitysettings. In the second exposure period, neither of the R and B pixelsare saturated. Thus, the final output ratio G:R:B becomes1.7:1.46:0.85=1:0.86:0.5 which is not deviated from the appropriateoutput ratio in the case illustrated in FIGS. 8A to 8E. Morespecifically, the imaging apparatus according to the present firstexemplary embodiment can prevent the coloring of the periphery of abright subject which moved.

Contrary to the case illustrated in FIGS. 11A to 11E, FIGS. 12A to 12Eillustrate the electric charge quantity of each pixel in the imagingapparatus according to the first exemplary embodiment in a case wherethe light quantity is an intermediate light quantity in the firstexposure period and exceeds the G saturation level in the secondexposure period. More specifically, FIGS. 12A to 12E illustrate anexample case where the region in the imaging plane changes from adarkish state to a bright state. Also, in this case, coloring can beprevented by suitably differentiating the saturation charge quantity ofthe photoelectric conversion portion PD between the G, R, and B pixelsas in the first exemplary embodiment.

FIGS. 13A to 13E illustrate a case where a subject perceived as white bythe human eyes is captured by the imaging apparatus according to thefirst exemplary embodiment under a different light source from the lightsource in the cases illustrated in FIGS. 8A to 12E. Also, in this case,like the cases illustrated in FIGS. 11A to 12E, the ratio of thesaturation charge quantity of the photoelectric conversion portion PD ofeach pixel is equalized to the white balance ratio under the lightsource of the imaging environment. In the case illustrated in FIGS. 13Ato 13E, under this light source, the light quantity ratio G:R:B when asubject perceived as white by the human eyes is captured, i.e., thewhite balance ratio is 1:0.5:0.5. Therefore, in the example illustratedin FIGS. 13A to 13E, when the saturation charge quantity of thephotoelectric conversion portion PD of the G pixel is set to 1, thesaturation charge quantity of the photoelectric conversion portion PD ofthe R pixel is set to 0.5, and the saturation charge quantity of thephotoelectric conversion portion PD of the B pixel is set to 0.5. Also,in this case, this configuration is implemented by suitably adjustingthe OFF-level potential of the gate drive line PTX3 for each of the G,R, and B pixels.

FIGS. 13A to 13E illustrate the electric charge quantity of each pixelin the imaging apparatus according to the first exemplary embodiment ina case where the light quantity exceeds the G saturation level in thefirst exposure period and becomes an intermediate light quantity in thesecond exposure period. Also, in this case, like the case illustrated inFIGS. 11A to 11E, when the G pixel is saturated in the first exposureperiod, the R and B pixels also reach respective saturation quantitysettings. In the second exposure period, neither of the R and B pixelsare saturated. Thus, the final output ratio G:R:B becomes1.6:0.8:0.8=1:0.5:0.5 which is not deviated from the appropriate outputratio. More specifically, the imaging apparatus of the present firstexemplary embodiment can prevent the coloring of the periphery of abright subject which moved.

Since the saturation charge quantity of the photoelectric conversionportion PD between pixels is differentiated in this way, the deviationof the pixel output ratio can be prevented in a case where the globalelectronic shutter operation for electric charge transfer from thephotoelectric conversion portion PD to the holding portion MEM isperformed during the exposure period for acquiring one frame. Morespecifically, the deviation of the pixel output ratio can be preventedby differentiating the saturation charge quantity of the photoelectricconversion portion PD between pixels according to color information (ortype information) of the color filter of each pixel and the whitebalance under the light source during imaging.

The imaging apparatus according to the present exemplary embodimentincludes a plurality of pixels PIX. The plurality of pixels PIX includesa first pixel and a second pixel each having a different saturationcharge quantity of the photoelectric conversion portion PD. In thiscase, it is desirable that the first and the second pixels havedifferent processing conditions by the signal processing unit in thesubsequent stage. It is also desirable that the signal processingincludes processing for multiplying the signal output from each pixelPIX by the gain and that the first and the second pixels are assigneddifferent gains. The gain is typically the white balance gain. Thisconfiguration enables preventing the deviation of the pixel outputratio. It can also be said that the first and the second pixels aredifferent in sensitivity.

As described above, the saturation charge quantity of the photoelectricconversion portion PD of each pixel can be controlled by the height ofthe potential barrier of the discharge switch TX3 of each pixel in theOFF state. In a case where the discharge switch TX3 of each pixel is atransistor, the height of the potential barrier of the discharge switchTX3 in the OFF state can be controlled by the OFF voltage to be appliedto the gate electrode of the discharge switch TX3, i.e., the OFF-levelpotential of the gate drive line PTX3. Alternatively, in a case wherethe discharge switch TX3 of each pixel is a transistor, the height ofthe potential barrier of the discharge switch TX3 in the OFF state canbe controlled by the impurity concentration in a semiconductor regiondisposed under the gate electrode of the discharge switch TX3.

It is desirable that the saturation charge quantity of the photoelectricconversion portion PD of each pixel can be changed. More specifically,it is desirable that at least either one of the saturation chargequantity of the photoelectric conversion portion PD of the first pixeland the saturation charge quantity of the photoelectric conversionportion PD of the second pixel can be changed. This makes it possible tochange the saturation charge quantity of the photoelectric conversionportion PD of each pixel according to imaging conditions and processingconditions in the signal processing unit, whereby the deviation of thepixel output ratio can be prevented under various conditions. In a casewhere the imaging apparatus includes a condition setting unit forsetting processing conditions in the signal processing unit, it isdesirable to change the saturation charge quantity of the photoelectricconversion portion PD of each pixel according to a setting by thecondition setting unit. Typically, it is desirable to change thesaturation charge quantity of the photoelectric conversion portion PD ofeach pixel according to a white balance setting by the condition settingunit. In this way, coloring can be prevented under different lightsources.

Although, in the examples illustrated in FIGS. 11A to 13E, the whitebalance ratio is equalized to the ratio of the saturation chargequantity of the photoelectric conversion portion PD of the G, R, and Bpixels, the configuration is effective even if complete matching is notmade. When a white subject is captured, if the saturation chargequantity of the photoelectric conversion portion PD of a pixel with highsensitivity is set larger than the saturation charge quantity of thephotoelectric conversion portion PD of a pixel with low sensitivity, aneffect of reducing coloring is provided. More specifically, if the firstpixel has a larger white balance gain in signal processing in thesubsequent stage than the second pixel, the saturation charge quantityof the photoelectric conversion portion PD of the first pixel needs tobe set smaller than the saturation charge quantity of the photoelectricconversion portion PD of the second pixel. Then, if the pixelsensitivity ratio (white balance ratio) is brought close to the ratio ofthe saturation charge quantity of the photoelectric conversion portionPD of each pixel, an effect of reducing coloring increases. To avoidextremely unnatural coloring, it is desirable to restrict the deviationbetween the pixel sensitivity ratio and the ratio of the saturationcharge quantity of the photoelectric conversion portion of each pixel to±30% or less. More specifically, it is desirable to satisfy thefollowing formula (1).0.7R≤r≤1.3R  (1)

Referring to formula (1), R is the ratio of the gain to be applied formultiplication to the signal of the second pixel to the gain to beapplied for multiplication to the signal of the first pixel in signalprocessing by the signal processing unit. Referring to formula (1), r isthe ratio of the saturation charge quantity of the photoelectricconversion portion PD of the second pixel to the saturation chargequantity of the photoelectric conversion portion PD of the first pixel.

Although FIGS. 10A to 13E illustrate cases where the incident lightquantity to the region changes near the boundary between the first andthe second exposure periods, the effects of the present exemplaryembodiment are not limited to the cases. Even in a case where theincident light quantity changes during the first or the second exposureperiod and a case where the light quantity changes not in step form butin slope form, similar effects are provided.

Although the imaging apparatus of the present exemplary embodimentoperates at least in the second drive mode, the drive mode may beswitchable and the imaging apparatus may be operable also in the firstdrive mode. In the first drive mode (one-time transfer operation), whenthe photoelectric conversion portion PD of the G pixel is saturated, theengine in the subsequent stage can determine the saturation andtherefore the coloring problem does not arise. Therefore, in the firstdrive mode, the saturation charge quantity of the photoelectricconversion portion PD may be identical for each pixel. Since the imagingapparatus according to the first exemplary embodiment is configured toseparately set the OFF-level potential of the gate drive line PTX3 ofeach pixel, the imaging apparatus can separately set the saturationcharge quantity of the photoelectric conversion portion PD of eachpixel. Thus, in response to the drive mode switching, the control unit 3may change the saturation charge quantity of the photoelectricconversion portion PD of each pixel. More specifically, when the firstdrive mode is selected, the control unit 3 may change the OFF-levelpotential of the gate drive line PTX3 of each pixel to be identical foreach pixel. If the timing of the control signal of the first transferswitch TX1 of each pixel is made variable and the OFF-level potential ofthe gate drive line PTX3 is changed based on the number of times oftransfer or transfer timing, these changes are supported within animaging apparatus.

A second exemplary embodiment will be described below. Only differencesfrom the first exemplary embodiment will be described in detail below.Like the first exemplary embodiment, an overall block diagram of theimaging apparatus according to the second exemplary embodiment isillustrated in FIG. 1 .

The imaging apparatus according to the second exemplary embodiment willbe described below centering on the second exemplary embodiment as anexample. FIG. 14 illustrates an equivalent circuit of the pixelsincluded in the imaging region 2 in the imaging apparatus according tothe second exemplary embodiment. As drawn by the thick lines illustratedin FIG. 14 , in the imaging apparatus according to the second exemplaryembodiment, the discharge switches TX3 of the R and B pixels are drivenby the same line, and the discharge switch TX3 of the G pixel (Gr, Gb)is driven by the same line. In the imaging apparatus according to thesecond exemplary embodiment, as illustrated in FIG. 14 , the drive lineof the discharge switch TX3 of each pixel PIX meanders according to thetype of the color filter CF. The gate drive line PTX(n) and the gatedrive line PTX(n+2) are identical in a region (not illustrated).

For example, in the example case under the light source according to thefirst exemplary embodiment illustrated in FIGS. 13A to 13E, thesensitivity (or the white balance gain) when a subject perceived aswhite by the human eyes is captured is identical for each of the R and Bpixels. Thus, coloring can be prevented if the saturation chargequantity of the photoelectric conversion portion PD of the G pixel andthe saturation charge quantities of the photoelectric conversionportions PD of the R and B pixels can be separately adjusted. In otherwords, in this case, coloring can be prevented even if the saturationcharge quantity of the photoelectric conversion portion PD of the Rpixel and the saturation charge quantity of the photoelectric conversionportion PD of the B pixel cannot be separately adjusted.

Like the circuit illustrated in FIG. 14 , the imaging apparatusaccording to the second exemplary embodiment separates the line for thedriving the discharge switch TX3 of the G pixel and the line for drivingthe discharge switches TX3 of the R and B pixels. This configurationenables separately adjusting the OFF-level potential of the dischargeswitch TX3 between the G, R, and B pixels. According to the secondexemplary embodiment, since the discharge switch TX3 of the R pixel andthe discharge switch TX3 of the B pixel are driven by the same line, thesaturation charge quantities of the R and B pixels cannot be separatelyadjusted. However, coloring can be prevented depending on the lightsource, for example, under the light source illustrated in FIGS. 13A to13E.

A third exemplary embodiment will be described below as another exampleof the imaging apparatus according to the second exemplary embodiment.FIG. 15 illustrates an equivalent circuit of the pixels included in theimaging region 2 in the imaging apparatus according to the thirdexemplary embodiment. As illustrated in FIG. 15 , in the imagingapparatus according to the third exemplary embodiment, the dischargeswitches TX3 of all of the G, R, and B pixels are driven by the sameline. The gate drive lines PTX(n), PTX(n+1), and PTX(n+2) are identicalin a region (not illustrated).

According to the third exemplary embodiment, the saturation chargequantity of the photoelectric conversion portion PD is differentiatedbetween pixels by differentiating, between pixels, the impurityconcentration of the semiconductor region disposed under the gateelectrode of the discharge switch TX3.

FIG. 16 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus according to the third exemplary embodiment.More specifically, FIG. 16 illustrates a cross-section of one pixel. Thecross-sectional structure of the imaging apparatus according to thepresent exemplary embodiment differs from the cross-sectional structureof the imaging apparatus according to the first exemplary embodimentillustrated in FIG. 3 in that a p-type region 214 as an individualsemiconductor region for each pixel is provided under the gate electrode211 of the discharge switch TX3 via an insulating layer (notillustrated). In the imaging apparatus according to the third exemplaryembodiment, the height of the potential barrier of the discharge switchTX3 in the OFF state is differentiated by differentiating the impurityconcentration of the p-type region 214 between the G, R, and B pixels.This configuration differentiates the saturation charge quantity of thephotoelectric conversion portion PD between the G, R, and B pixels toequalize the ratio of the saturation charge quantity of each pixel tothe white balance ratio. With this arrangement, coloring is prevented.

A third exemplary embodiment will be described below. Only differencesfrom the first exemplary embodiment will be described in detail below.Like the first exemplary embodiment, an overall block diagram of theimaging apparatus according to the third exemplary embodiment isillustrated in FIG. 1 .

FIG. 17 illustrates an equivalent circuit of the pixels included in theimaging region 2 in the imaging apparatus according to the thirdexemplary embodiment. The imaging apparatus according to the thirdexemplary embodiment includes normal pixels and infrared (IR) pixels. AnIR pixel refers to a pixel having a larger depth of the photoelectricconversion portion PD in a semiconductor substrate than a normal pixelto provide high sensitivity to infrared light. The imaging apparatusaccording to the third exemplary embodiment does not have color filters.Each normal pixel of the imaging apparatus according to the thirdexemplary embodiment has a similar structure to each pixel according tothe first and the second exemplary embodiments except that a colorfilter is not provided.

As illustrated in FIG. 17 , in the imaging apparatus according to thethird exemplary embodiment, IR pixels are arranged in the nth row,normal pixels are arranged in the (n+1)th row, and IR pixels arearranged in the (n+2)th row. An output containing only the infraredlight component can be obtained from the imaging apparatus according tothe third exemplary embodiment by subtracting the output from a normalpixel from the output from an IR pixel in adjacent rows (for example,the nth and (n+1)th rows).

FIG. 18 is a schematic diagram illustrating a cross-sectional structureof the imaging apparatus according to the third exemplary embodiment.More specifically, FIG. 18 illustrates a cross-section of one IR pixel.In the cross-sectional structure of a normal pixel, the color filter CFhas been removed from the structure illustrated in FIG. 3 . Instead ofthe N-type semiconductor region 202 included in a normal pixel, an IRpixel includes an N-type semiconductor region 216 having a large depthin a semiconductor substrate.

Second Comparative Example

Prior to the description of a configuration and effects of the presentexemplary embodiment, another comparative example will be describedbelow. In the imaging apparatus according to the second comparativeexample, a normal pixel and an IR pixel have the same saturation chargequantity of the photoelectric conversion portion PD. FIGS. 19A to 20Dillustrate operations of the imaging apparatus according to the secondcomparative example under different incident light conditions. FIGS. 19Ato 21D are similar to FIGS. 8A to 13E. FIGS. 19A, 20A, and 21A aretiming charts illustrating drive pulses. FIGS. 19B, 20B, and 21Billustrate the light quantity of light incident to a certain region ofthe imaging apparatus. FIGS. 19C, 20C, and 21C illustrate the changeswith time of the electric charge quantity of the photoelectricconversion portion PD of an IR pixel. FIGS. 19D, 20D, and 21D illustratethe changes with time of the electric charge quantity of thephotoelectric conversion portion PD of a normal pixel.

In the situation illustrated in FIGS. 19A to 19D, the photoelectricconversion portion PD of neither of the normal and the IR pixels issaturated in both the first and the second exposure periods. Therefore,the ratio of the outputs from the IR and normal pixels for the frame isthe same as the sensitivity ratio of both pixels. The output from the IRpixel is 0.8+0.8=1.6, the output from the normal pixel is 0.5+0.5=1, andthe output ratio (IR pixel to normal pixel) is 1.6:1.

FIGS. 20A to 20D are diagrams illustrating an example case where theincident light quantity changes in the first and the second exposureperiods according to the second comparative example. In the situationillustrated in FIGS. 20A to 20D, the IR pixel is saturated in the firstexposure period and is not saturated in the second exposure period. Onthe other hand, the normal pixel is not saturated in the first and thesecond exposure periods. Therefore, the final pixel output ratio willdeviate from the pixel sensitivity ratio. Since the output of the IRpixel is 1+0.8=1.8 and the output of the normal pixel is0.875+0.5=1.375, the final pixel output ratio (IR pixel to normal pixel)becomes 1.8:1.375=1.309:1 which is deviated from the sensitivity ratio1.6:1. If the IR pixel output minus the normal pixel output iscalculated, there is obtained an inaccurate image which contains lessinfrared light component than that in a case where the IR pixel is notsaturated in each exposure period as illustrated in FIGS. 19A to 19D.More specifically, a deviation of the pixel output ratio occurs even inthis case.

In an imaging apparatus according to a fourth exemplary embodiment as anexample of the third exemplary embodiment, the saturation chargequantity of the photoelectric conversion portion PD is differentiatedbetween the IR and the normal pixels. More specifically, the imagingapparatus includes a first pixel and a second pixel each having adifferent saturation charge quantity of the photoelectric conversionportion PD. More specifically, when the saturation charge quantity ofthe photoelectric conversion portion PD of the IR pixel is set to 1, thesaturation charge quantity of the photoelectric conversion portion PD ofthe normal pixel is set to 0.625. As illustrated in FIG. 17 , thisconfiguration is implemented by separately providing the line of thegate drive line PTX3 for the IR and the normal pixels anddifferentiating the OFF-level potential of the gate drive line PTX3between the IR and the normal pixels.

Like the case illustrated in FIGS. 20A to 20D, FIGS. 21A to 21Dillustrate the electric charge quantity of each pixel in the imagingapparatus according to the fourth exemplary embodiment in a case wherethe light quantity exceeds the IR saturation level in the first exposureperiod and becomes an intermediate light quantity in the second exposureperiod. As described above, the saturation charge quantity isdifferentiated between the IR and the normal pixels based on thesensitivity ratio. Therefore, when the IR pixel is saturated in thefirst exposure period, the normal pixel also reaches the set saturationquantity. In the second exposure period, neither of the IR and normalpixels are saturated. As a result, the final output ratio (IR pixel tonormal pixel) becomes 1.8:1.125=1.6:1 which is not deviated from theappropriate output ratio in the case illustrated in FIGS. 19A to 19D.

As described above, in the imaging apparatus according to the thirdexemplary embodiment, the saturation charge quantity of thephotoelectric conversion portion PD is differentiated between pixels.This configuration enables preventing the deviation of the pixel outputratio in a case where the global electronic shutter operation forelectric charge transfer from the photoelectric conversion portion PD tothe holding portion MEM is performed during the exposure period foracquiring one frame.

The imaging system according to the fourth exemplary embodiment will bedescribed below with reference to FIG. 22 . FIG. 22 is a block diagramillustrating an overall configuration of the imaging system according tothe present exemplary embodiment.

The imaging apparatuses according to the first to the third exemplaryembodiments are applicable to diverse imaging systems. Examples of suchimaging systems are not particularly limited and include digital stillcameras, digital camcorders, monitoring cameras, copying machines,facsimiles, mobile phones, on-vehicle cameras, and observationsatellites. A camera module including an optical system (such as lenses)and an imaging apparatus is also included in an imaging system. FIG. 22is a block diagram of a digital still camera as an example of an imagingsystem.

As illustrated in FIG. 22 , an imaging system 300 includes an imagingapparatus 100, an imaging optical system 302, a central processing unit(CPU) 310, a lens control unit 312, an imaging apparatus control unit314, an image processing unit 316, a diaphragm shutter control unit 318,a display unit 320, an operation switch 322, and a recording medium 324.

The imaging optical system 302 forms an optical image of a subject andincludes a lens group and a diaphragm 304. The diaphragm 304 is providedwith a function of adjusting the light quantity at the time of imagecapturing by adjusting the aperture diameter, and a function of anexposure time adjustment shutter used at the time of still imagecapturing. The lens group and the diaphragm 304 is held in such a mannerthat the lens group and the diaphragm 304 can retract along the opticalaxis direction, so that a zooming function and a focus adjustmentfunction can be implemented by a collaborative operation of the lensgroup and the diaphragm 304. The imaging optical system 302 may beintegrated with the imaging system 300 and may be an imaging lensattachable to the imaging system 300.

The imaging apparatus 100 is disposed in such a manner that the imagingplane thereof is located in the image space of the imaging opticalsystem 302. The imaging apparatus 100 is any one of the imagingapparatuses according to the first to the third exemplary embodimentsand includes a CMOS sensor (a pixel region 10) and a peripheral circuit(peripheral circuit region). In the imaging apparatus 100, a pluralityof pixels each having a photoelectric conversion portion istwo-dimensionally arranged, and a color filter is disposed in each ofthese pixels to configure a two-dimensional single-plate color sensor.The imaging apparatus 100 performs the photoelectric conversion on asubject image formed by the imaging optical system 302 and outputs animage signal and a focus detection signal.

The lens control unit 312 controls forward/backward drive of the lensgroup of the imaging optical system 302 to perform a zooming operationand focus adjustment. The lens control unit 312 includes circuits andprocessing units for implementing these functions. The diaphragm shuttercontrol unit 318 adjusts the imaging light quantity by changing theaperture diameter of the diaphragm 304 (with a variable diaphragmvalue). The diaphragm shutter control unit 318 includes circuits andprocessing units for implementing these functions.

The CPU 310, a control unit for managing various control of the cameramain body, includes a calculation unit, a read only memory (ROM), arandom access memory (RAM), an analog-to-digital (A/D) converter, adigital-to-analog (D/A) converter, and a communication interfacecircuit. The CPU 310 controls operations of each section in the cameraaccording to a computer program held in the ROM to perform a series ofimaging operations including auto focus (AF) including detection of thefocus state of the imaging optical system 302 (focus detection), imagecapturing, image processing, and recording. The CPU 310 also serves as asignal processing unit.

The imaging apparatus control unit 314 controls operations of theimaging apparatus 100, performs the A/D conversion on the signal outputfrom the imaging apparatus 100 and transmits the resultant signal to theCPU 310. The imaging apparatus control unit 314 includes circuits andcontrol units configured to implement these functions. The imagingapparatus 100 may be provided with the A/D conversion function. Theimage processing unit 316 performs image processing such as gammaconversion and color interpolation on the signal having undergone theA/D conversion to generate an image signal. The image processing unit316 includes circuits and control units configured to implement thesefunctions. As described above, the imaging system 300 includes aprocessing apparatus including the CPU 310 and the image processing unit316. The processing apparatus performs various correction and datacompression processing on the imaging data output from the imagingapparatus 100.

The display unit 320 is a display apparatus such as a liquid crystaldisplay (LCD) for displaying information about the imaging mode of thecamera, a preview image before image capturing, an image for checkingafter image capturing, and the in-focus state at the time of focusdetection. The operation switch 322 includes a power switch, a release(imaging trigger) switch, a zoom operation switch, and an imaging modeselection switch. The recording medium 324 records a captured image, andmay be a medium built in the imaging system 300 or a medium such as amemory card that can be attached and detached to and from the imagingsystem 300.

With the above described configuration, the imaging system 300 to whichany one of the imaging apparatus 100 according to the first to the thirdexemplary embodiments is applied becomes possible to implement ahigh-performance imaging system capable of performing high-precisionfocus adjustment and acquiring an image with a large depth of field.

An imaging system and a mobile object according to a fifth exemplaryembodiment will be described below with reference to FIGS. 23A and 23B.FIGS. 23A and 23B are diagrams illustrating configurations of theimaging system and the mobile object according to the present exemplaryembodiment.

FIG. 23A illustrates an example of an imaging system 400 related to anon-vehicle camera. The imaging system 400 includes an imaging apparatus410 which is any one of the above-described imaging apparatusesaccording to the first to the third exemplary embodiments. The imagingsystem 400 includes an image processing unit 412 for performing imageprocessing on a plurality of pieces of image data acquired by theimaging apparatus 410, and a parallax acquisition unit 414 forcalculating the parallax (parallax image phase difference) based on theplurality of pieces of image data acquired by the imaging apparatus 410.The imaging system 400 also includes a distance acquisition unit 416 forcalculating the distance to a subject based on the calculated parallax,and a collision determination unit 418 for determining the possibilityof collision based on the calculated distance. The parallax acquisitionunit 414 and the distance acquisition unit 416 are examples of adistance information acquisition unit for acquiring information aboutthe distance to a subject. More specifically, the information about thedistance is information about the parallax, defocusing amount, and thedistance to a subject. The collision determination unit 418 maydetermine the possibility of collision by using these pieces of thedistance information. The distance information acquisition unit may alsobe implemented by a specially designed hardware component or a softwaremodule. The distance information acquisition unit may also beimplemented by a Field Programmable Gate Array (FPGA), an ApplicationSpecific Integrated Circuit (ASIC), or a combination of both.

The imaging system 400, connected with the vehicle informationacquisition apparatus 420, acquires vehicle information including thevehicle speed, yaw rate, and steering angle. The imaging system 400 isalso connected with a control ECU 430 as a control apparatus foroutputting control signals for generating a braking force to a vehiclebased on the determination result by the collision determination unit418. More specifically, the control ECU 430 is an example of a mobileobject control unit for controlling a mobile object based on thedistance information. The imaging system 400 is also connected with analarm apparatus 440 for generating an alarm to a driver based on thedetermination result by the collision determination unit 418. Forexample, when the possibility of collision is determined to be high bythe collision determination unit 418, the control ECU 430 performsvehicle control for avoiding a collision and reducing damages byapplying brakes, returning the accelerator, and restricting the enginepower. The alarm apparatus 440 warns a user by generating an alarm suchas sound, displaying alarm information on the screen of a car navigationsystem, or vibrating the seat belt and steering.

According to the present exemplary embodiment, the imaging system 400captures an image of the periphery of the vehicle, such as the vehiclefront or back. FIG. 23B illustrates the imaging system 400 for capturingan image of the vehicle front (an imaging range 450). The vehicleinformation acquisition apparatus 420 activates the imaging system 400and transmits an instruction thereto to perform image capturing. Byusing any one of the above-described imaging apparatuses according tothe first to the third exemplary embodiments as the imaging apparatus410, the imaging system 400 according to the present exemplaryembodiment can further improve the accuracy of distance measurement.

Although the present exemplary embodiment has been described abovecentering on an example of control for avoiding a collision with othervehicles, the present exemplary embodiment is also applicable toautomatic driving control for following another vehicle and automaticdriving control for retaining the vehicle within a lane. The imagingsystem 400 is applicable not only to vehicles such as automobiles butalso to other mobile objects (moving apparatuses) such as vessels,airplanes, and industrial robots. In addition, the imaging system 400 isapplicable not only to mobile objects but also to intelligent transportsystems (ITS) and a wide range of apparatuses utilizing objectrecognition.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An imaging apparatus comprising: a plurality ofpixels, including a first pixel and a second pixel, each including aphotoelectric conversion portion configured to accumulate electriccharges generated by incident light, a holding portion configured tohold the electric charges, an amplification portion configured to outputa signal based on the electric charges, a first transfer switchconfigured to transfer the electric charges from the photoelectricconversion portion to the holding portion, and a second transfer switchconfigured to transfer the electric charges from the holding portion tothe amplification portion; output lines connected to the plurality ofpixels, and a signal processing unit configured to perform signalprocessing on signals each of which is the signal and which are outputfrom the plurality of pixels based on any one of a plurality ofprocessing conditions, wherein the imaging apparatus has a first drivemode and a second drive mode, the first drive mode and the second drivemode being able to be switched, wherein, in the first drive mode, at afirst time, the photoelectric conversion portions of the plurality ofpixels start accumulating the electric charges, from the first time to asecond time, the first transfer switch of at least one of the pluralityof pixels is kept OFF, and the photoelectric conversion portion of theat least one of the plurality of pixels accumulates electric chargesgenerated in a first period that starts at the first time and ends atthe second time, at the second time, the first transfer switches of theplurality of pixels are controlled to turn ON from OFF, and at a thirdtime following the second time, the holding portions of the plurality ofpixels hold both electric charges generated in the photoelectricconversion portion in the first period, and electric charges generatedin the photoelectric conversion portion in a second period that startsat the second time and ends at the third time, wherein, in the seconddrive mode, when an exposure period for acquiring one frame starts, thephotoelectric conversion portions of the plurality of pixels startaccumulating the electric charges, from a time when the exposure periodstarts until the exposure period ends, the first transfer switch of atleast one of the plurality of pixels is kept OFF, when the exposureperiod ends, the first transfer switches of the plurality of pixels arecontrolled to turn ON from OFF, and wherein, in response to switchingbetween the first drive mode and the second drive mode, saturationcharge quantities of the photoelectric conversion portions of the atleast one of the plurality of pixels are changed, and wherein, in thesignal processing, a processing condition applied to the signal outputfrom the first pixel and a processing condition applied to the signaloutput from the second pixel are different from each other.
 2. Theimaging apparatus according to claim 1, wherein the imaging apparatuscaptures images for a first frame and a second frame following the firstframe, an exposure period corresponding to the second frame includes thefirst period and the second period, at the first time, the holdingportion holds electric charges corresponding to the first frame, and inthe first period, the second transfer switches of the plurality ofpixels turn ON in turns and the amplification portions of the pluralityof pixels output the signals in turns in such a manner that outputs ofthe signals to be used in the image for the first frame are completed bythe second time at the latest.
 3. The imaging apparatus according toclaim 1, wherein the signal processing includes processing formultiplying the signal by a gain, and in the signal processing, a firstgain to be applied in amplifying the signal output from the first pixeland a second gain to be applied in amplifying the signal output from thesecond pixel are different from each other.
 4. The imaging apparatusaccording to claim 3, wherein the first gain and the second gain are setin accordance with a white balance.
 5. The imaging apparatus accordingto claim 3, wherein in the signal processing, the first gain is smallerthan the second gain, and a saturation charge quantity of thephotoelectric conversion portion of the first pixel is larger than asaturation charge quantity of the photoelectric conversion portion ofthe second pixel.
 6. The imaging apparatus according to claim 3,wherein, in the signal processing, the following formula is satisfied:0.7R≤r≤1.3R where r is a ratio of the second gain to the first gain, andR is a ratio of the saturation charge quantity of the photoelectricconversion portion of the second pixel to the saturation charge quantityof the photoelectric conversion portion of the first pixel.
 7. Theimaging apparatus according to claim 1, wherein at least either one of asaturation charge quantity of the photoelectric conversion portion ofthe first pixel and a saturation charge quantity of the photoelectricconversion portion of the second pixel is configured to be variable. 8.The imaging apparatus according to claim 1, further comprising acondition setting unit configured to set the plurality of processingconditions in the signal processing unit, wherein, according to asetting of the plurality of processing conditions by the conditionsetting unit, the imaging apparatus varies at least either one of thesaturation charge quantity of the photoelectric conversion portion ofthe first pixel and the saturation charge quantity of the photoelectricconversion portion of the second pixel.
 9. An imaging apparatuscomprising: a plurality of pixels, including a first pixel and a secondpixel, each including a photoelectric conversion portion configured toaccumulate electric charges generated by incident light, a holdingportion configured to hold the electric charges, an amplificationportion configured to output a signal based on the electric charges, afirst transfer switch configured to transfer the electric charges fromthe photoelectric conversion portion to the holding portion, and asecond transfer switch configured to transfer the electric charges fromthe holding portion to the amplification portion; and output linesconnected to the plurality of pixels, wherein the imaging apparatus hasa first drive mode and a second drive mode, the first drive mode and thesecond drive mode being able to be switched, wherein, in the first drivemode, at a first time, the photoelectric conversion portions of theplurality of pixels start accumulating the electric charges, from thefirst time to a second time, the first transfer switch of at least oneof the plurality of pixels is kept OFF, and the photoelectric conversionportion of the at least one of the plurality of pixels accumulateselectric charges generated in a first period that starts at the firsttime and ends at the second time, at the second time, the first transferswitches of the plurality of pixels are controlled to turn ON from OFF,and at a third time following the second time, the holding portions ofthe plurality of pixels hold both electric charges generated in thephotoelectric conversion portion in the first period, and electriccharges generated in the photoelectric conversion portion in a secondperiod that starts at the second time and ends at the third time, andwherein, in the second drive mode, when an exposure period for acquiringone frame starts, the photoelectric conversion portions of the pluralityof pixels start accumulating the electric charges, from a time when theexposure period starts until the exposure period ends, the firsttransfer switch of at least one of the plurality of pixels is kept OFF,when the exposure period ends, the first transfer switches of theplurality of pixels are controlled to turn ON from OFF, and wherein, inresponse to switching between the first drive mode and the second drivemode, saturation charge quantities of the photoelectric conversionportions of the at least one of the plurality of pixels are changed,wherein each of the plurality of pixels further includes a dischargeswitch configured to discharge the electric charges accumulated in thephotoelectric conversion portion, and a height of a potential barrierprovided by a discharge switch of the first pixel in an OFF state isdifferent from a height of a potential barrier provided by a dischargeswitch of the second pixel in an OFF state.
 10. The imaging apparatusaccording to claim 9, wherein at least either one of the height of thepotential barrier provided by the discharge switch of the first pixel inan OFF state and the height of the potential barrier provided by thedischarge switch of the second pixel in an OFF state is configured to bevariable.
 11. The imaging apparatus according to claim 9, wherein thedischarge switch is a transistor, and the height of the potentialbarrier provided by the discharge switch in an OFF state varies inaccordance with a voltage to be applied to a gate of the dischargeswitch for setting the discharge switch in the OFF state.
 12. Theimaging apparatus according to claim 9, wherein the discharge switch isa transistor, and an OFF voltage to be applied to a gate of thedischarge switch of the first pixel is different from an OFF voltage tobe applied to a gate of the discharge switch of the second pixel. 13.The imaging apparatus according to claim 9, wherein the discharge switchis a transistor, and an impurity concentration of a semiconductor regionthat is disposed under a gate electrode of the discharge switch of thefirst pixel is different from an impurity concentration of asemiconductor region that is disposed under a gate electrode of thedischarge switch of the second pixel.
 14. An imaging apparatuscomprising: a plurality of pixels, including a first pixel and a secondpixel, each including a photoelectric conversion portion configured toaccumulate electric charges generated by incident light, a holdingportion configured to hold the electric charges, an amplificationportion configured to output a signal based on the electric charges, afirst transfer switch configured to transfer the electric charges fromthe photoelectric conversion portion to the holding portion, and asecond transfer switch configured to transfer the electric charges fromthe holding portion to the amplification portion; and output linesconnected to the plurality of pixels, wherein the imaging apparatus hasa first drive mode and a second drive mode, the first drive mode and thesecond drive mode being able to be switched, wherein, in the first drivemode, at a first time, the photoelectric conversion portions of theplurality of pixels start accumulating the electric charges, from thefirst time to a second time, the first transfer switch of at least oneof the plurality of pixels is kept OFF, and the photoelectric conversionportion of the at least one of the plurality of pixels accumulateselectric charges generated in a first period that starts at the firsttime and ends at the second time, at the second time, the first transferswitches of the plurality of pixels are controlled to turn ON from OFF,and at a third time following the second time, the holding portions ofthe plurality of pixels hold both electric charges generated in thephotoelectric conversion portion in the first period, and electriccharges generated in the photoelectric conversion portion in a secondperiod that starts at the second time and ends at the third time,wherein, in the second drive mode, when an exposure period for acquiringone frame starts, the photoelectric conversion portions of the pluralityof pixels start accumulating the electric charges, from a time when theexposure period starts until the exposure period ends, the firsttransfer switch of at least one of the plurality of pixels is kept OFF,when the exposure period ends, the first transfer switches of theplurality of pixels are controlled to turn ON from OFF, wherein, inresponse to switching between the first drive mode and the second drivemode, saturation charge quantities of the photoelectric conversionportions of the at least one of the plurality of pixels are changed, andwherein each of the first and the second pixels includes a color filterthat is disposed on the photoelectric conversion portion, and a color ofthe color filter of the first pixel is different from a color of thecolor filter of the second pixel.
 15. The imaging apparatus according toclaim 14, wherein the color filter included in the first pixel is agreen color filter, the color filter included in the second pixel is ared color filter or a blue color filter, and a saturation chargequantity of the photoelectric conversion portion of the first pixel islarger than a saturation charge quantity of the photoelectric conversionportion of the second pixel.
 16. An imaging apparatus comprising: aplurality of pixels, including a first pixel and a second pixel, eachincluding a photoelectric conversion portion configured to accumulateelectric charges generated by incident light, a holding portionconfigured to hold the electric charges, an amplification portionconfigured to output a signal based on the electric charges, a firsttransfer switch configured to transfer the electric charges from thephotoelectric conversion portion to the holding portion, and a secondtransfer switch configured to transfer the electric charges from theholding portion to the amplification portion; and output lines connectedto the plurality of pixels, wherein the imaging apparatus has a firstdrive mode and a second drive mode, the first drive mode and the seconddrive mode being able to be switched, wherein, in the first drive mode,at a first time, the photoelectric conversion portions of the pluralityof pixels start accumulating the electric charges, from the first timeto a second time, the first transfer switch of at least one of theplurality of pixels is kept OFF, and the photoelectric conversionportion of the at least one of the plurality of pixels accumulateselectric charges generated in a first period that starts at the firsttime and ends at the second time, at the second time, the first transferswitches of the plurality of pixels are controlled to turn ON from OFF,and at a third time following the second time, the holding portions ofthe plurality of pixels hold both electric charges generated in thephotoelectric conversion portion in the first period, and electriccharges generated in the photoelectric conversion portion in a secondperiod that starts at the second time and ends at the third time,wherein, in the second drive mode, when an exposure period for acquiringone frame starts, the photoelectric conversion portions of the pluralityof pixels start accumulating the electric charges, from a time when theexposure period starts until the exposure period ends, the firsttransfer switch of at least one of the plurality of pixels is kept OFF,when the exposure period ends, the first transfer switches of theplurality of pixels are controlled to turn ON from OFF, wherein, inresponse to switching between the first drive mode and the second drivemode, saturation charge quantities of the photoelectric conversionportions of the at least one of the plurality of pixels are changed, andwherein each of the plurality of pixels includes a color filter that isdisposed on the photoelectric conversion portion and a discharge switchfor discharging the electric charges accumulated in the photoelectricconversion portion, the plurality of pixels are arranged in a matrixform such that the first pixel and the second pixel are arranged in asame row/column, a color of the color filter of the first pixel isdifferent from a color of the color filter of the second pixel, and atleast two lines are provided in the same row/column, one connected tothe discharge switch of the first pixel and another one connected to thedischarge switch of the second pixel.
 17. The imaging apparatusaccording to claim 1, wherein each of the plurality of pixels includes acolor filter that is disposed on an upstream side of the photoelectricconversion portion in an incident direction of the incident light, and adischarge switch for discharging the electric charges accumulated in thephotoelectric conversion portion, pixels having different types of thecolor filters are arranged in a matrix form, and based on a type of thecolor filter, wiring of a gate of the discharge switch for each of theplurality of pixels is arranged in a winding manner.
 18. An imagingapparatus comprising: a plurality of pixels, including a first pixel anda second pixel, each including a photoelectric conversion portionconfigured to accumulate electric charges generated by incident light, aholding portion configured to hold the electric charges, anamplification portion configured to output a signal based on theelectric charges, a first transfer switch configured to transfer theelectric charges from the photoelectric conversion portion to theholding portion, and a second transfer switch configured to transfer theelectric charges from the holding portion to the amplification portion;output lines connected to the plurality of pixels, and a setting unitconfigured to set a white balance, wherein the imaging apparatus has afirst drive mode and a second drive mode, the first drive mode and thesecond drive mode being able to be switched, wherein, in the first drivemode, at a first time, the photoelectric conversion portions of theplurality of pixels start accumulating the electric charges, from thefirst time to a second time, the first transfer switch of at least oneof the plurality of pixels is kept OFF, and the photoelectric conversionportion of the at least one of the plurality of pixels accumulateselectric charges generated in a first period that starts at the firsttime and ends at the second time, at the second time, the first transferswitches of the plurality of pixels are controlled to turn ON from OFF,and at a third time following the second time, the holding portions ofthe plurality of pixels hold both electric charges generated in thephotoelectric conversion portion in the first period, and electriccharges generated in the photoelectric conversion portion in a secondperiod that starts at the second time and ends at the third time,wherein, in the second drive mode, when an exposure period for acquiringone frame starts, the photoelectric conversion portions of the pluralityof pixels start accumulating the electric charges, from a time when theexposure period starts until the exposure period ends, the firsttransfer switch of at least one of the plurality of pixels is kept OFF,when the exposure period ends, the first transfer switches of theplurality of pixels are controlled to turn ON from OFF, wherein, inresponse to switching between the first drive mode and the second drivemode, saturation charge quantities of the photoelectric conversionportions of the at least one of the plurality of pixels are changed, andwherein, according to a setting of the white balance, the imagingapparatus varies at least either one of the saturation charge quantityof the photoelectric conversion portion included in the first pixel andthe saturation charge quantity of the photoelectric conversion portionincluded in the second pixel.
 19. An imaging apparatus comprising: aplurality of pixels each including a photoelectric conversion portionconfigured to accumulate electric charges generated by incident light, aholding portion configured to hold the electric charges, anamplification portion configured to output a signal based on theelectric charges, a first transfer switch configured to transfer theelectric charges from the photoelectric conversion portion to theholding portion, a second transfer switch configured to transfer theelectric charges from the holding portion to the amplification portion,and a discharge switch configured to discharge the electric chargesaccumulated in the photoelectric conversion portion; and output linesconnected to the plurality of pixels, wherein at a first time, thephotoelectric conversion portions of the plurality of pixels startaccumulating the electric charges, from the first time to a second time,the first transfer switch of at least one of the plurality of pixels iskept OFF, and the photoelectric conversion portion of the at least oneof the plurality of pixels accumulates electric charges generated in afirst period that starts at the first time and ends at the second time,the first transfer switches of the plurality of pixels are controlled toturn ON from OFF by the second time at the latest, at a third timefollowing the second time, the holding portions of the plurality ofpixels hold both electric charges generated in the photoelectricconversion portion in the first period, and electric charges generatedin the photoelectric conversion portion in a second period that startsat the second time and ends at the third time, the discharge switch is atransistor, and an impurity concentration of a semiconductor region thatis disposed under a gate electrode of the discharge switch of the firstpixel is different from an impurity concentration of a semiconductorregion that is disposed under a gate electrode of the discharge switchof a second pixel.
 20. The imaging apparatus according to claim 19,wherein each of the first pixel and the second pixel includes a colorfilter that is disposed on the photoelectric conversion portion, and acolor of the color filter of the first pixel is different from a colorof the color filter of the second pixel.
 21. The imaging apparatusaccording to claim 19, wherein the imaging apparatus captures images fora first frame and a second frame following the first frame, an exposureperiod corresponding to the second frame includes the first period andthe second period, at the first time, the holding portion holds electriccharges corresponding to the first frame, and in the first period, thesecond transfer switches of the plurality of pixels turn ON in turns andthe amplification portions of the plurality of pixels output the signalsin turns in such a manner that outputs of the signals to be used in theimage for the first frame are completed by the second time at thelatest.
 22. The imaging apparatus according to claim 19, furthercomprising a signal processing unit configured to perform signalprocessing on the signals each of which is the signals and which areoutput from the plurality of pixels based on any one of a plurality ofprocessing conditions, wherein, in the signal processing, a processingcondition applied to the signal output from the first pixel and aprocessing condition applied to the signal output from the second pixelare different from each other.
 23. The imaging apparatus according toclaim 22, wherein the signal processing includes processing formultiplying the signal by a gain, and in the signal processing, a firstgain to be applied in amplifying the signal output from the first pixeland a second gain to be applied in amplifying the signal output from thesecond pixel are different from each other.
 24. The imaging apparatusaccording to claim 23, wherein the first gain and the second gain areset in accordance with a white balance.
 25. The imaging apparatusaccording to claim 23, wherein in the signal processing, the first gainis smaller than the second gain, and a saturation charge quantity of thephotoelectric conversion portion of the first pixel is larger than asaturation charge quantity of the photoelectric conversion portion ofthe second pixel.
 26. The imaging apparatus according to claim 23,wherein, in the signal processing, the following formula is satisfied:0.7R≤r≤1.3R where r is a ratio of the second gain to the first gain, andR is a ratio of the saturation charge quantity of the photoelectricconversion portion of the second pixel to the saturation charge quantityof the photoelectric conversion portion of the first pixel.
 27. Theimaging apparatus according to claim 19, wherein each of the pluralityof pixels further includes a discharge switch configured to dischargethe electric charges accumulated in the photoelectric conversionportion, and a height of a potential barrier provided by a dischargeswitch of the first pixel in an OFF state is different from a height ofa potential barrier provided by a discharge switch of the second pixelin an OFF state.
 28. The imaging apparatus according to claim 27,wherein at least either one of the height of the potential barrierprovided by the discharge switch of the first pixel in an OFF state andthe height of the potential barrier provided by the discharge switch ofthe second pixel in an OFF state is configured to be variable.
 29. Theimaging apparatus according to claim 27, wherein the discharge switch isa transistor, and the height of the potential barrier provided by thedischarge switch in an OFF state varies in accordance with a voltage tobe applied to a gate of the discharge switch for setting the dischargeswitch in the OFF state.
 30. The imaging apparatus according to claim27, wherein the discharge switch is a transistor, and an OFF voltage tobe applied to a gate of the discharge switch of the first pixel isdifferent from an OFF voltage to be applied to a gate of the dischargeswitch of the second pixel.
 31. The imaging apparatus according to claim19, wherein each of the first and the second pixels includes a colorfilter that is disposed on the photoelectric conversion portion, and acolor of the color filter of the first pixel is different from a colorof the color filter of the second pixel.
 32. The imaging apparatusaccording to claim 31, wherein the color filter included in the firstpixel is a green color filter, the color filter included in the secondpixel is a red color filter or a blue color filter, and a saturationcharge quantity of the photoelectric conversion portion of the firstpixel is larger than a saturation charge quantity of the photoelectricconversion portion of the second pixel.
 33. The imaging apparatusaccording to claim 19, wherein each of the plurality of pixels includesa color filter that is disposed on the photoelectric conversion portionand a discharge switch for discharging the electric charges accumulatedin the photoelectric conversion portion, the plurality of pixels arearranged in a matrix form such that the first pixel and the second pixelare arranged in a same row/column, a color of the color filter of thefirst pixel is different from a color of the color filter of the secondpixel, and at least two lines are provided in the same row/column, oneconnected to the discharge switch of the first pixel and another oneconnected to the discharge switch of the second pixel.
 34. The imagingapparatus according to claim 19, wherein each of the plurality of pixelsincludes a color filter that is disposed on an upstream side of thephotoelectric conversion portion in an incident direction of theincident light, and a discharge switch for discharging the electriccharges accumulated in the photoelectric conversion portion, pixelshaving different types of the color filters are arranged in a matrixform, and based on a type of the color filter, wiring of a gate of thedischarge switch for each of the plurality of pixels is arranged in awinding manner.
 35. An imaging system comprising: the imaging apparatusaccording to claim 1; and a processing apparatus configured to processesa signal output from the imaging apparatus.
 36. An imaging systemcomprising: the imaging apparatus according to claim 19; and aprocessing apparatus configured to processes a signal output from theimaging apparatus.
 37. A mobile object comprising: the imaging apparatusaccording to claim 1; a distance information acquisition unit configuredto acquire information about a distance to a subject based on a signaloutput from the plurality of pixels of the imaging apparatus; and acontrol unit configured to control the mobile object based on thedistance information.
 38. A mobile object comprising: the imagingapparatus according to claim 19; a distance information acquisition unitconfigured to acquire information about a distance to a subject based ona signal output from the plurality of pixels of the imaging apparatus;and a control unit configured to control the mobile object based on thedistance information.
 39. An imaging system comprising: the imagingapparatus according to claim 9; and a processing apparatus configured toprocesses a signal output from the imaging apparatus.
 40. An imagingsystem comprising: the imaging apparatus according to claim 14; and aprocessing apparatus configured to processes a signal output from theimaging apparatus.
 41. An imaging system comprising: the imagingapparatus according to claim 16; and a processing apparatus configuredto processes a signal output from the imaging apparatus.
 42. An imagingsystem comprising: the imaging apparatus according to claim 18; and aprocessing apparatus configured to processes a signal output from theimaging apparatus.
 43. A mobile object comprising: the imaging apparatusaccording to claim 9; a distance information acquisition unit configuredto acquire information about a distance to a subject based on a signaloutput from the plurality of pixels of the imaging apparatus; and acontrol unit configured to control the mobile object based on thedistance information.
 44. A mobile object comprising: the imagingapparatus according to claim 14; a distance information acquisition unitconfigured to acquire information about a distance to a subject based ona signal output from the plurality of pixels of the imaging apparatus;and a control unit configured to control the mobile object based on thedistance information.
 45. A mobile object comprising: the imagingapparatus according to claim 16; a distance information acquisition unitconfigured to acquire information about a distance to a subject based ona signal output from the plurality of pixels of the imaging apparatus;and a control unit configured to control the mobile object based on thedistance information.
 46. A mobile object comprising: the imagingapparatus according to claim 18; a distance information acquisition unitconfigured to acquire information about a distance to a subject based ona signal output from the plurality of pixels of the imaging apparatus;and a control unit configured to control the mobile object based on thedistance information.